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ABSTRACT ■ , . . 

- ‘ . "Good teaching" is notoriously difficult . to measure. 

It may be possible, though, to , say where .a teacher stands on -a scale 
of development which has four /stages. In yie first stage, the 
authority is the teacher, and/ the student gives back rote responses. 
In the secbbd stage, the ..authpri^\y is the text or syllabus, and the . 
student asks "questions oiily to clarify the assignment or get a fact. 
In .jtoe third stage, right ans/wers ^re derived from evidence rather 
than^ the text, and the student; begins to discuss and ask thoughtfiiL 
questions.. In the fourth, thp student decides^for himself what the' 
authority is and spends his time inj individual work. Que.stions which 
may gauge the stage of development, pf a teacher include how much the 
teacher talks, what kind of.quiestiohs he asks, what kind of questions 
the. student asks, what kind'" of study materials ai^ja used, .how closely 
the teacher follows the lhssc|n ^uide, if any, and how many topics 
refer to the student's own environment. (JK) . > ' ^ 
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• .. ^ Research Memorand'um No. 1 .* 

■» ■ ■ ■ ■ i 

This is one pf a series of reports of. research on 
the Educational Reform Program -of El 'Salvador, and- ' 

■ especially its -use of instructional televifsion. 

This report has been prepared by members of the 
Institute for Communication Research, StanfoW, 
University, on behalf of the Academy for Educational 
Development i under contract vrith- the JJ.S. Agency for 
International Development, 
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, These notes, grew out ofijur thinking about research ; on. El . . . 


" 


i Salvador's broad program of educational reform, of ;>5hich instructional ' ' 

television &nd teacher in-service training are among the major compo- , , . o 

' .'.ij^nts. Because the ideas may be . useful elsewhere, they have been set ' - r. - 


0 / 


\ ’ • ... • ' . ■ • . ■ * ■ . * 

/down "in this memorahdum. . 

We are making a deljLberately naive approach tp a very complex*^ / ' 

subject. Throughout all the histpry of education there has been no ^ ^ 




. broad agreement on what constitutes good teaching or a good teacher. ' 
One reason for this is that "there are many ‘outcomes of education alhd 




different ways of achieving fhem. Some are difficult to measure if v 




\ they can be measured at . all, and' others cannot be known until long 

afterward. A judgment on good teaching therefore often involves us ^ 


f 


^ a . • 

in value judgments in an area where the hard evidence is often insuf- : ' ‘ 

ficient.' How^ shall we evaluate, for example, a teacher who by example 


1 

1 * ^ 


Al^arently teaches us honesty and kindness, but little arithmetic, ^ 


! n 


againsb another teacher who teaches us to add, .subtract, multiply, • ' i 

^ 4 * 

and divide, but little about human decency? HOw shall" we evaluate a 

* t * . • ! ■ 

teacher who makes us miserable in high school mathematics, although ^ 

• . j ■ ■ •i’ 

some of us look back in later years with the greatest of- gratitude ( 


' ^ 


for that stern pedagogy? How shall we evaluate a teacher who system- . . 1 • 

• ‘ \ : 

atically and clearly covers the assigned subject matter, against • . ; 

- , - . • . ‘ ^ ^ ' 1 
anothei^who is unsystematic and fails to cover important areas but j 

' ' ‘ " 'i 

imparts some intellectual curiosity? How shall we evaluate a Socrates ’ \ 


O ' ' 
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who could provide uneqyalled challenge to highly superior students , | 

against a teacher who would be of no use to Socrates* students but \ ’** 

, " • . . » -1 • 
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> is higliJiA^fective withNretarded or deprived students^ Questions 

. . J. ■ ■ Cl • 

like this have always^^^lrobd • in' the way of measuring "good" teaching. 

/ ' i ' 

The subject therefore has usually been talked about in rather general 
terms, and more often in the spirit of philosophy and exhortatiBW than 
in the spirit of science. . . 

Why try. to measure -jit all; wha% happens in the classroom?. IThy 
not be content with measuring the end product -- what the student 
learns, and how he changes in the course’ of being taught, -at least 
insofar as those results , can be measured? The reason is that we • ^ 

\ a <■ . • 

should like to know what kind of teaching, vihat. kinds of cla'Ssropm 'h 
experience, bring" about different learning results. • We should like 
• to know how to train teachers. • And, .'especially in developing- coun-’ 

. tries pnd developing school systems, we should .like to be able to 

* ’ * ■ ■ • ' 

help teachers climb the ladder toward modern pedagogy, just. as. we • 

4 .*• .A a f . 

help farmers, technicians, managers, and doctors move toward modern ' 

\ ■ ■ ■ ■ 



^ills and practices. > ‘ *-i • 

’ There ire, to be 'sure, sdme excellent instrument's fbr • - 

,i> . . . \ • 

' ■ * ' a «> , ■ 

describing the interaction between teacher and pupils in ai classroom. 

The Flanders measures are;, an outstanding example. Thefe- are also 

■ ■ ■ . V ■ 

numerous evaluative instruments, among them the IOTA (Instrument for 

Observation of Teacher Activity) and -jthe’ Robertson Teacher Self- 

, » 

* \ , 

» ' . ' o , 

Appraisal System. Typically these require highly expert obse'lrvers, 
who need a nunlber of’ hours of training before they can achieve inter- 

V 

observer reliability.^. They are not the Icind of tools that couljd be 
used readily by a school supervisor or a utilization qfficfer trying 
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to. fielp a teachei* in ’a 'deyeloping scho^ system. Furthermore, they 

aremade for,- and fit better, the schools of economically advanced 

. » ' • 

, countries than, those of developing countries. . 

•. Suppose we begi<n, riot by trying to measure the "goodness’’ of 
• teaching, but.j rathe^ by trying to say where it Stands on a scale of 
development whiqh /schoijls are believed to go through as they move 
from the earliest stage the child learning from the parent or the 
'elder whrit they learned from their parents or their elders --to the 

? r ' • . ' 

-/most modern, where the emphasis is^:gn the child learning to seek and 

solve problems that are relevant t^him, working at hip own'pace, ^ 
*• - ■’ ^ -* 
with the whole machinery df the school being used to stimulate and 

6 1 - 

support rather ^han regulate him. Somebody has called this the Path 

">• r^’ '* 'I 

. •* , • * 

from Wallaby to Winrietka. ' ' - . " 

• One typology of^ this kind has .been. presented by C.,F.^Beeby 

’ , • '' ■ ' ’ ' 

it\ his book, The_pualitv of Education in Developing Countries , to 

» , ' * * 

which we gratefully acknowledge a'debt. .The^pattern we are going to* 
suggest is much like fh^t of Beeby's, biit differs from it in certain 
important respects, and we^shall move on, as Beeby does ;jlot, to 
inquire how progifess-^along. the path could be measured. 

What is the p^th that at) educational system follows as it 

moves fi^'om Vlallaby to VJinnetka? . * . ' 

' ■ . * ■■ ■ ' . 1 

' . 4'' . 

In the earliest tiuman societies children learned at their 

* *N. . • * * 

'parents’ knees. Their mothers passed on to them the legends and 
customs of their people, and thelf • fal^hers taught them to hunt or 
fish. After -a tipe some of these duties >ere^ taken over by some of 






the most expert .^mcftig the adults -- the best storytellers, the priest, » 
the best hupters or warriors. When labor came to be more and more 
differentiated, the. role of teacher was recognized. 'As Ihte .as 1800 
in the United Staf^ and Europe, many schools st;Lll existed in which . 
untrained teachers^passed on. to young children the rudiments of 
counting and writing and spelling, taught them the favorite poems 
of earlier generations, and the favorite stories out of the national 
histoi^r -- just as the teacher had learned them, and in the same way. 
This is what Beeby. calls the '’dpme school"; We have never .seen 
schools of this type in El Salvador, but have seen ^hem in ^rtain , , , 
other developing countries. 

The next stage in development comes when the .curriculum is 
.formalized with a syllabus. The teacher is better trained than the' ' 
"dame school*^ teacher, but not well enough trained to be confident 
of his ability to depart from the syllabus, or from t]he drills and 
teaching suggestions contained in his teachers' guide. Schools like 
this are very common in developing countries, El Salvador included,. 

At this' stage of develbpment, the syllabus acquires a degree of 

<j * * ' • * 

f ' ' ■ * < ‘ 

authority that might startle the people who wrote it — principally 
because the examinations are based .upon it, .and upon the examinations 
often -depend further schooling and careers. This is particularly true* 
of some former colonies. Many a Peace Corps Volunteer^ who has tried 
to liven i^p a course in Africa by digressing frop the topic has. been 
startled to hear his students shout at' him, "N.O.S.I" — meaning, "Not 
on. Syllabus-,." and therefore a. waste of time. Similarly, the teachers' 
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gui^de takes on an authority that sometimes surprises the authors of • 
the guide, principally because the teacher is uncomfortable dej^rtiQ^g 
frdm it. When television is introduced into a develo.ping country,/ 
many classroom teachers have to deal both with. unfamiliar subject 

matter and unfamiliar teaching methods, and consequently hold on for 

’ < ■ • ■ 

dear life to their syllabus and guide. 

■ "N' : 

' At this stage 'of development, the student does not do anything 
mqch different from what he did at the "dame school" stag^. He listens, 

• ' . f ' 

he drills,^ he gives the teacher back- whaf he is expected to, and he 

chdnts many of his responses. The chanting drill, .in whichs-the class 

responds in a common rhythm almost like a choTjps, is heard all through 
. ' / • 
the. developing countries. The' student, at this stage, does not ask 

many questions,' and almiost never ventures an opinion. 

One, more step up the ladder, 9'id a great change becomes visible 

• , ■■ ■ " j ' _ ■ 

in the classroom. -No longer is the teacher so poorly prepared, that, he 
feels ill at ease with new math or the second language and must hold 
on to the syllabus as to a llf£ preserver. No longer is he so little 
trained that he must parrot the exercises he hfas been given in the . 

o . • . _ 

same way as he expects his pupils to parrot ,the answers.. He feels 
free to vary from the syllabus' when necessary in order to make the 
points apply to the experiences and interests and future needs of 
his students. He feels able to introduce new classroom activities 

If- 

of *his 'Own, and tries to enrich the school time with” learning aids 

,, . 'a 0 I 

and class projects. He. feels able to aok questions that have more 
than a single right answer, and so some of the question-^and-answer' 
time turns into discussion, rather than drill. 
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'An even more spectacular changfe comes. oVer the students' at 

, ■ ' * . . , ' *' * ' '** ■ ^ . " . * . ' v.^ 

this stage. In earlier stages they have^been silent, for the most 

; , • J ■ * ■ ’ i 

part, except yhen called upon to recite or to, join in a drill. If. 
they have asked questions, i^ has usually been for instruiftental 
reasons -- what did you tell us to do. Miss ^o and So? tihere can - . 
I find a ruler? May I leave the room? -- or tp clarify something — 
what did you tell us to do? Now they begin to ask thoughtful ques- 
tions, --..what is the meaning of- this? Is this really true? T^at 
would h^pen if the conditions were .so and so? They begin to relate 
their own experiences to what they are learning in class, and/.feel 
for the first time that \ijhat they have, seen aijd done- may be worth 



vcy^c 



talking about, and <th6ir ‘opinions ‘sometimes worth voacing. Bit by, 

• ..r. \ ■ • 

'bit, the locus of -authority in the. classroom begins to change from 
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, the teacher ai(d the syllabus or thxt to the consideration of evidence 

‘ '1 '*» ... 

in the environment^ Th^’ ’function of the class is no longefr to^learn 

by rote what the dame schopl teacher gives them, or to le^n ta give 

- ' ■ ■ ' - ■ ‘ 

back the syllabus, but rather to apply these Vpoints to the world they 

•. know, and even sometimes to challenge them. As' this takes place, the 

• . . ;V . 

chanting drills come to be heard less and less often, and are replaced 
by class discussion and student projects. , • V 

In a fourth stage, the teacher is well informed, well trained, 

■ confident ‘in his. ability to’ euicTK .'individual students alocie the paths 

■ / to learnl'Pg that seem most promising for them. The spirit of^ the- : 

/ . '■ ^ ' / : 

. . classroom now is that an individual must decide for himself, nn’ the 

basis of, the best evidence availalile, where authority lies in any ‘ . 
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given are^. of knowledge. ’Not all the answers have been found and 
• _ .some of- the o^ld answers . are q^^i^e^io'nable; knowledge is still to be 

sought'. The learnings experiences become individualized. Not that ; 

* .-S ^ 

lectures or drills cease to play a part, in the classroom process, 

I 

but increasingly the studei|ts take responsibility, for their own * • 

. r ' * 

learning, and under the teacher's guidance move forward at their 

L -V ■ 

own pace, often working on projects and problems of*, their own choice. 

. f 

Discussion is even more important than in the previous'’stage-. ^ Stu- 

■ ■ • ■ • '■ / ’ ' ■ . 

^ dents of ten challenge .statements heard or read/ in. class . The class- 

room is no. longer an echo cliamber for the teacher, as in the first 

stage, or a controlled ejcercise in rote learning, as in stage two, 

^ ' i V 

but rather a situation in which the teacher is a senior prartner in - 

4 ' • 

the ^kperience of learning, but every member of the class is also a 
full par tnei;^ responsible for his own progress, for his own drill 
(perhaps with the aid of programmed instruction or CAT), and for . 

contributing to the learning experiences bf others. And as this 

! ■ ■ . ■ ■ 

happens some o£ the time-hbnqred accountrements of schools) begin 
• ■ ■ 1 . . ■ ■ . • \ 

. • ; r , ■ , • . 

to disappear --^'the di^vision into grade levels, the rigidly scheduled 

o • ■ * ■ . 

^.day, the frequent givj^pg of letter grades, and so forth. 

We have tried to summarize these stages in the foltowing 
• - chart: / * • , ' . ’ ■„ . • 
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If stage four seems idealistic, it is. Not many schpols, 
even in 'highly Industrialized countries, have reached It. There are 

, ^ ti 

• . Q ’ 

schools, in remote -areas and less-developed countries, still in stage 
one. Most schools 'in El SalV|adbf are- probably in stage two pr . stage 
three, and theemoyement is from two to three. Most 'schools in the 
.United States are probably in. sCage three, with th'e movement toward 
four. . , ' J' ' 1 , 

The\m6vtfment of schools along this path of development is • 

' ■ f\‘' ■ - ■ '' 

apparently irreversible, -barring' -some great catastrophe tjiat would- - 
destroy much W. the culture! It reminds';6ne of Whitehead 's^whimsical 
statement in explaining Newton's Seconld Law, that the only way one 
could tell whether -a movie of the universe were being run b'acWwerd 
or forward is to notice whether entropy increases, for the Second 

\ ^ f ^ ' • 

Law. shows that entropy always increases with time. ! Similatly, a 

' * . ■ . 

.* I . . ' ' ' 

developing School system, so far as we h^ve been able to observe, . 
always moves away frpm what we have described as stage one in* the 
direction bf stage fou^. ' V ^ ^ . 

^ ' It may b^n^sj.eading to describe the path as four stages. 

Certainly, a cjoilntr;jj or a school does not take three giant steps'. > 
from .stage one to stage.^fout. It takes many tiny’ steps„ and progress 

' ■ • ' ■ '• " ■ ' J * 

is often slow Snd uneven; The four stagers we have suggested’ ^ar« only^ 

po^-nts on a continuum. \ ' • . ' ' •’ 

. Vj ' . ' ' ■ ' ' 

. An expert observer^ given a typology ’like the one v;d have 
' p.resented' in this memorandu\n, could say in genersl terms whether a 
classr' is functioning approximately on the 'level of stage one. 



stage two, stage three, or stage four. , However, we need measurements, 
that can be applied by observers who ar£ not great experts’, and we 
need something both more: reliable and more sensitive than a general 
conclusion that a’ teach e'r 'and' a class are about a.t stage two. We. 
need to be able to fecogniEe changes that take place in a relatively' 
short time. -We need to know whether a teacher and hi-s class are 
\ moving, along the desired path, and where progress lags, if it ‘does. 

What can we obser’ve in the classroom to answer some^ of .those 
needs? More speclficaHji': What are the items that can be'.pbserved 

reliably by supeirvisofs who are not really trained as’; rese^chers? - 
These restrictions eliminate some of. the possibilities. As 

V . fi- ' 

attractive as it would be tp obtain a Rampling of class time'-- for; 
example, checking off every three sec^ds what is going on in the 

-4 " ' , * . ■" ^ . ' ■ , « 

classroom, as some of the Instruments do we'^are.led to believe^* 
that this cannig^t be expected of supervisors who are' not trained as ■ 
researchers. At least so we have been advised; if that is 'incorrect;. 



we c.an then build an instrii^ent on a time base.; Again, it Would be 



■N 



very, hard to obtain feliablelqualitative judgments of some things it 
might be useful to know- -- jSuch as the students' apparent l^el of 
interest,- tjie teacher's, ability to maintain atte'^ntion, or. his skill 

.’■•■.O'." ■ ” ' -i 

at' sensing, the readiness or the optimum perfo'rmante. level of his ;: - 

students. It is not impoasible to think of making an objective , 

■'measure, of interest or attention, but that is a later and longer* 

• ■ ■ ■ , \ •' 

task. \ ■ . ' ' : ■ 



What, then, can we tr.easure2r We shall have to find out partly’ 

* '* ^ V a' ' ^ 

hy trial and error... But here are. S fev^shgge/tions. * • 



In the first \ 5 lace, how much the teacher talks as compared 

■' ■ - ' ■ ■ i 

to how much, the students talk or v7ork on^their own tasks. |n 

9 4 

1 * ' , 

Salvador, the Plant- Basico classes using the old curriculum without . 
television are mostly lecture. When television is introdj^ced, that 

t * 

is largeljy lecture, and the pre- and post-television times are used 
chiefly fo.r drill or further exposition by the teacher. As a class- 
room moves toward the third stage, , however, it is reasonable to 
suppose that the proportion of student talk would increase. Ideally 

this measure requires a time sampling, but the differences appear tb\ 

• \ ■■ X. 

be so gross that a five-po:|nt or seven-poin^ observational scale "Hs 

worth trying, to see vjhether it will yield usdful results. 

■ r r f) ' - 

' ' ■ I V ' ' 

What kinds - of questions doe's the teacher ask ? The proportion 



of single-answer questions (drill) to multiple-answer, questions 
-(thought and discussion)- should increase dramatically as a classroom 
.moves upward, especially from the second stage to the third. 

* ■ , , ■ c 

. What kinds of questions - does the student ask ? Ouif observation 

is that children in the first two stages, if they ask any questions at 

“all, ask largely Instrumental ones such as, what did you say the 
\' • * • 

assignment was? or, where is the chalk? As they move through the, 
second stage and into the. third, however, they begin to ask, first, 
clarifica^on questions ^(what does this mean?), and then thought' 
questions (what would .happen if X-., ^ .. ? or, does this^ule really 

apply to ?). Therefore the number ^fxjue^^ons of these 

three different kinds asked by. students during a class hour"“should 
be a revealing measure. 



What kind s of study materials are used in the class? At the 



first stage, there is likely to be none; the teacher is the source 



/ 



of knowledg^f^ In the second stage, we, are likely to find a s^llabu 
or a text; and as 'the class moves upward from the second stage the 
.numl>er of learning rescJ^i\fes increases significantly. It should be 
possible to make a list of these resources that could be checked off 

for any given class. These would include the learning aids -- pic- 

^ ■ ' * 

tures, maps, charts, demonstrations^, and the like that usually 

begin to appear in stage two, increase' greatly in stage three, and 
are individualized in stage four. , ” 

‘ . r r 

How cl6,selv does the teacher follow the lesson guide , if any? 
Inability to foU-oif it is characteristic of pre-stage-two teaching, 

and. ability to vary from It so 'as to develop new exercises or appli- 

\ ■ • ' ■ 

\ ... 

cations that are designed especially to fit the students in a 
particular class is characteristic of. post- stage-two teaching. ]^t 
should be possible, therefore, to set up a sdties of judgments based 
on_ knowledge of the lesson guide: does, the' teacher fall short of _ 

covering the topics adequately? does he cover the^ topics and.exe'r- 

• * ■ ' ' * . ' ■ 

cises adequately? does he go beyond adequacy in the way described 

above? %t may well be possible, by experimenting with measures^ of 
this kind, co obtain quite reliable observations. 

How many-feopics examples , materials used during the cla%s . 
hour specifically' refer rto. the s tudent * s own environment and experi- 
ence? This number might be expected to Increase steadily from stages 

■ n . * • 

one to four. ' It 'should be possible to devise a list of different 



\ 

kinds of, materials and experience (e.g., examples in the lecture 
materralj problems, visuals, demonstrations, etc.) under which locally 
.oriented’ items could be checked as they occur ^ • 

What is the proportion of discussion to drill and/or recitation ? 
Here,„ as in some 6f the cases above, it .would.be best to have a time 

sampling, but the differenc.es are likely to^ be so gross, and discussion 

so rare in the first two stages, that ^ five or seven point scale might 
prove to be feasible. . ^ • 

How many class projects are In evidence ? How many individual 

. - ' ■ t' 

• ones ? One would expect these numbers to increase, and the proportion 
of individual to slags work to Increase. • ' 

How many examples of individualization can be observed during , 
the class hour? For example, does the teacher comment on individuar 
papers or exercises? Does he spend time with individual students? 

How many students cite- their own experiences in class discussion? 

How many students seem to be wbrklng on the^r own projects?’ How' 






13 , 



rfia , 

many are working on problems or practicing by themselves during the 
class hour? How many seem to be working at ■ their own pace without 
ybeipg locked into the class. rate of progress? These numbers should 
fncrease, especially as the class 'moves toward the 'fourth stage of 
development. It would seem possible^ for an observer to' answer each 

* ^ . p' ■ 

of these questions either with a number (how many students cite 

* < . • / 

I * i H " t . 

• th§ir own experiences? or even with- arf. all -or -non^ measure: does 

any student cite his own;. experience? does the teacher . spdnd any 
time with individual students?). It might be well to^begin with all- 




/ 



■ ■ . . , 14 

■ . ' * 

o'r-none measures, , and later to refine somp of. them to provide for 
counting. ' ' 

. There are-many other possibilities, ,and the instrument used 

' ■ V' 

in El Salvador or elsewhere will have to meet, the -particular edtica- 
tipnal .purposes'and goals of the users. Therefore, these are' merely 
sugges't;ions toward an instrument, which will have to grow out of 

fur tjier thought, and out of trial by the kinds of observers who are 

. ' ■ * 

expecteM to use it. But such an effort is well worth making. 





